Affordance based Part Recognition for Grasping and Manipulation
نویسندگان
چکیده
Affordances (unit utility, functional and topological relationships) and semantic scene understanding are key to building a generic, scalable and cognitive architecture for visual perception. ‘Affordance based object recognition’ or recognition based on affordance features is an important step in this regard. In this paper, we extend the scope of affordance features to define ‘Conceptual Equivalence Classes’ and to recognize these classes leading to a Visual Cognitive Engine. This schema is analogous to building a scalable unit (part/ part assembly/ object) recognition system. This system has been built with the target of enabling a robotic arm to grasp a-priori unseen objects in a cluttered scene using global knowledge. The Visual Cognitive Engine uses inputs from multiple ontology bases – ConceptNet (for unit concept definitions), WordNet/WebKB (for textual unit definitions), Otto Bock Human Grasping database (for grasp affordances), together with the Part Functional Affordance Schema, developed in this paper, to establish semantic unit affordances. Recognition of parts or part assemblies based on affordances, using the Visual Cognitive Engine enables knowledge of affordance and interaction modes for the entire object. This leads to a goaldirected object recognition and manipulation system that can perform implicit cognitive tasks such as substitute a cup for a mug, bottle, jug, pitcher, pilsner, beaker, chalice, goblet or any other unlabeled object, but with a physical part affording the ability to hold liquid and a part affording grasping, given the goal of ‘bringing an empty cup’ and no cups are available in the work environment. The performance of such a system is superior to traditional view-point based object recognition and manipulation systems. Keywords-Affordance, Recognition by Components, Part affordances, Grasping, Symbol binding
منابع مشابه
A System for Affordance Based learning of Object Grasping in a Robot
A system is described which takes synergies extracted from human grasp experiments and maps these onto a robot vision and hand-arm platform to facilitate the transfer of skills [1]. This system forms part of a framework which is extended by adding a self organizing map based affordance learning system. This affordance system learns the correlations between perceived object features and relevant...
متن کاملRobotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching
This paper presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the system is that it handles a wide range of object categories without needing any task-specific training data for novel objects. To achieve this, it first uses a categoryagnostic affordance prediction algorithm to select...
متن کاملRelational Affordance Learning for Task-Dependent Robot Grasping
Robot grasping depends on the specific manipulation scenario: the object, its properties, task and grasp constraints. Object-task affordances facilitate semantic reasoning about pre-grasp configurations with respect to the intended tasks, favouring good grasps. We employ probabilistic rule learning to recover such object-task affordances for task-dependent grasping from realistic video data.
متن کاملA Visual Servoing System for Interactive Human-Robot Object Transfer
As the demand for close cooperation between human and robots grows, robot manufacturers develop new lightweight robots, which allow for direct human-robot interaction without endangering the human worker. However, enabling direct and intuitive interaction between robots and human workers is still challenging in many aspects, due to the nondeterministic nature of human behavior. This work focuse...
متن کاملHand Motion and Grasping: Capturing, Recognizing and Synthesizing
In this paper we review the various previous methods in vision-based hand motion analysis and hand motion synthesis and grasping. We present our two solutions to the problem. First, we present an articulated hand pose estimation scheme, in which we improve the regression forest based methods by incorporating high-level hand part feature and hand motion constraints. Second, we present a framewor...
متن کامل